Back Propagation Fails to Separate Where Perceptrons Succeed
نویسندگان
چکیده
-It is widely believed that the back propagation algorithm in neural networks, for tasks such as pattern classification, overcomes the limitations of the perceptron. We construct several counterexamples to this belief. We also construct linearly separable examples which have a unique minimum which fails to separate two families of vectors, and a simple example with four two-dimensional vectors in a single layer network showing local minima with a large basin of attraction. Thus back propagation is guaranteed to fail in the first, and likely to in the second, example. We show that even multilayered (hidden layer) networks can also fail in this way to classify linearly separable problems. Since our examples are all linearly separable, the perceptron would correctly classify them. Our results disprove the presumption, made in recent years, that, barring local minima, back propagation will find the best set of weights for a given problem.
منابع مشابه
Training Multilayer Perceptrons with the Extende Kalman Algorithm
A large fraction of recent work in artificial neural nets uses multilayer perceptrons trained with the back-propagation algorithm described by Rumelhart et. a1. This algorithm converges slowly for large or complex problems such as speech recognition, where thousands of iterations may be needed for convergence even with small data sets. In this paper, we show that training multilayer perceptrons...
متن کامل4 . Multilayer perceptrons and back - propagation
Multilayer feed-forward networks, or multilayer perceptrons (MLPs) have one or several " hidden " layers of nodes. This implies that they have two or more layers of weights. The limitations of simple perceptrons do not apply to MLPs. In fact, as we will see later, a network with just one hidden layer can represent any Boolean function (including the XOR which is, as we saw, not linearly separab...
متن کاملPhysical modelling of caving propagation process and damage profile ahead of the cave-back
The cavability assessment of rock mass cavability and indicating the damage profile ahead of a cave-back is of great importance in the evaluation of a caving mine operation, which can influence all aspects of the mine operation. Due to the lack of access to the caved zones, our current knowledge about the damage profile in caved zones is very limited. Among the different approaches available, p...
متن کاملShort-term and Medium-term Gas Demand Load Forecasting by Neural Networks
The ability of Artificial Neural Network (ANN) for estimating the natural gas demand load for the next day and month of the populated cities has shown to be a real concern. As the most applicable network, the ANN with multi-layer back propagation perceptrons is used to approximate functions. Throughout the current work, the daily effective temperature is determined, and then the weather data w...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2004